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Title: MULTIMODAL NEUROELECTRIC INTERFACE DEVELOPMENT 

Leonard J. Trejo, Kevin R. Wheeler, Charles C. Jorgensen 

Abstract (paper presentation) 

This project aims to improve performance of NASA missions by developing multimodal neuroelectric technologies for augmented 
human-system interaction. Neuroelectric technologies will add completely new modes of interaction that operate in parallel with 
keyboards, speech, or other manual controls, thereby increasing the bandwidth of human-system interaction. We recent demonstrated 
| the feasibility of real-time electromyographic (EMG) pattern recognition for a direct neuroelectric human-computer interface. We j 
i recorded EMG signals from an elastic sleeve with dry electrodes, while a human subject performed a range of discrete gestures. A i 
i machine-learning algorithm was trained to recognize the EMG patterns associated with th^gestures and map them to control signals. ! 
Successful applications now include piloting two Class 4 aircraft simulations (F-15 and 757) and entering data with a "virtual" 
numeric keyboard. Current research focuses on on-line adaptation of EMG sensing and processing and recognition of continuous 
gestures. We are also extending this on-line pattern recognition methodology to electroencephalographic (EEG) signals. This will 
allow us to bypass muscle activity and draw control signals directly from the human brain. Our system can reliably detect p-rhythm ; 

' (a periodic EEG signal from motor cortex in the 10 Hz range) with a lightweight headset containing saline-soaked sponge electrodes, j 
The data show that EEG p-rhythm can be modulated by real and imaginary motions. Current research focuses on using biofeedback j 
to train of human subjects to modulate EEG rhythms on demand, and to examine interactions of EEG-based control with EMG-based j 
| and manual control. I 
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aircraft instrumentation . 
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heavy items. Provides for training exoskeleton 
to do tasks autonomously. 


Electrode Types & Locations 
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• Joystick- four electrode pairs on forearm 

• Typing - eight electrode pairs on forearm 


Hidden Markov Models 



bj(O) = 2 c jm N [0,n jm , 2 jm ] , mixture model 


Hidden Markov Model Overview 
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HMM Initialization 





Visualization & Understanding 



Time 
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Typing Data (cont’d) 
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gesture X can be compared with all time slices and all channels 
for gesture Y. 


Mutual Information (STSC) 
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- Space suits 

- Mobile applications 


Neuroelectric Interfaces 
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Demonstrate 2D BCI for data visualization control 


Biophysical Basis of EEG-based Control 

(Desynchronization of n-rhythm) 



(Adapted from Beatty, 1995) 





Pilot Study: Detection of \i- and (3-rhythms 


E T3 P 


C 2 

E lu 


<r % 
di E 


O o 

3 I 


LU <0 

I < 


<3 o 
o 5 

Q- O 
LU CL 


desyncrhonization (ERD) 


Sensorphone 



Recording 



Card Sorting Task 

—EG at C3 & £4 Bandpass 6-12 Hz. One epoch pre- and post- motion. 



Time (s) 


17.5H 



11.5 1,4.0 16.5 19.0 21.5 24.0 



Reaching Task - Eleven Motion Epochs 
8.7 Hz ERD at C3 & C4 from -3 to +5 sec relative to onset of motion 



Time (s) 








Simulated BCI for Moving Map Control 





ERD in Real Motion 
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Relatively smaller change in power 
Dynamics appear suitable for 2D display control 



